Method for processing vr audio and corresponding equipment

ABSTRACT

A method for processing a virtual reality (VR) audio and a corresponding equipment are provided. The method includes acquiring, by a transmitting terminal of a VR audio, an ambisonics signal rotation angle, wherein the ambisonics signal rotation angle is determined according to a first equipment rotation angle corresponding to a receiving terminal of the VR audio, rotating an ambisonics signal according to the acquired ambisonics signal rotation angle, and/or, acquiring, by the transmitting terminal of the VR audio, an order of a mixed order ambisonics (MOA) signal determined according to related information of the VR audio, and extracting an MOA signal from the ambisonics signal according to the order of the MOA signal. Accordingly, an ambisonics signal rotation angle according to a rotation angle of an equipment is determined, the rotation occurs, and an MOA signal is extracted.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. § 119to Chinese Patent Application No. 201710056192.1, filed on Jan. 25, 2017in the State Intellectual Property Office of the People's Republic ofChina, the disclosure of which is incorporated by reference herein inits entirety.

TECHNICAL FIELD

The disclosure relates to the technical field of virtual reality (VR)audios. More particularly, the disclosure relates to a method forprocessing a VR audio and a corresponding equipment.

BACKGROUND

As people pay more attention to virtual reality (VR) products, manycompanies and organizations focus on the development of VR technologies.The VR audio is a key technology in the VR field. The VR audio canprovide a user with auditory contents having a spatial resolution, sothat the user can have immersive VR application experience. The sense ofimmersion can be realized only if both the visual sense and the auditorysense are consistent with the real world, as shown in FIG. 1.

FIG. 1 is a schematic diagram of experience of a virtual reality (VR)audio according to the related art.

The VR content source is an issue concerned by many users at present. Toenable a user to experience rich VR applications and VR contents, anonline virtual content platform becomes a future development trend, anda user can use a VR equipment to browse VR contents on the onlinevirtual content platform in real time. However, the bandwidth use in thebrowsing process is an issue to be considered.

As one key technology in the existing VR audio, the ambisonicstechnology records and restores a physical sound field by sound fieldharmonic decomposition and successive approximation. Ambisonics usesspatial harmonics as independent signals. For L-order spatialambisonics, (L+1)² independent spatial harmonic signals are required, anarray consisting of (L+1)² microphones is at least required for pickup,and at least (L+1)² loudspeakers are required for playback. If the orderof an ambisonics signal is higher, the approximation effect of thespatial sound field is better. Therefore, a higher-order ambisonicssignal has a better spatial resolution. However, the bandwidth occupancysharply increases with the increase of the order.

FIG. 2 is a schematic diagram of the spatial resolution of ambisonicssound fields of different orders according to the related art.

FIG. 3A is a schematic diagram of a 3-order ambisonics sound field,where 16 independent signals are required, according to the related art.

To solve the problem in the ambisonics technology that the bandwidthoccupancy sharply increases with the increase of the order, a mixedorder ambisonics (MOA) technology has been proposed. In the MOAtechnology, different orders are used for sound fields in a horizontaldirection and a vertical direction. When a user gazes at a horizontalplane, the ears are differently sensitive to the sound in the horizontaldirection and the sound in the vertical direction, and are moresensitive to the sound in the horizontal direction. Therefore, contentsin the horizontal direction are transmitted at a higher order so thatthe contents in the horizontal direction have a high spatial resolution,meanwhile, contents in the vertical direction are transmitted at a loworder, thereby reducing the bandwidth occupancy.

FIG. 3B is a schematic diagram of an MOA sound field according to therelated art.

Referring to FIG. 3B, the horizontal direction is at a 3-order (a3-order two-dimensional ambisonic signal is used in the horizontaldirection), the vertical direction is at a 1-order (a 1-orderthree-dimensional ambisonic signal is used in the vertical direction).When the MOA technology is used, only 8 independent signals arerequired, and the bandwidth occupancy is equivalent to ½ of that for theambisonics technology.

However, the existing MOA technology is still not high enough in thespatial resolution accuracy but too high in the bandwidth occupancy.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providea method for processing a virtual reality (VR) audio and a correspondingequipment.

In accordance with an aspect of the disclosure, a method for processinga VR audio is provided. The method includes acquiring, by a transmittingterminal of a VR audio, an ambisonics signal rotation angle, determiningthe ambisonics signal rotation angle according to a first equipmentrotation angle corresponding to a receiving terminal of the VR audio,and rotating an ambisonics signal according to the acquired ambisonicssignal rotation angle.

In accordance with another aspect of the disclosure, a transmittingterminal equipment for a VR audio is provided. The transmitting terminalequipment includes an acquisition device configured to acquire anambisonics signal rotation angle, the ambisonics signal rotation anglebeing determined according to a first equipment rotation anglecorresponding to a receiving terminal of the VR audio, and a rotationdevice configured to rotate an ambisonics signal according to theambisonics signal rotation angle.

In accordance with another aspect of the disclosure, a method forprocessing a VR audio is provided. The method includes acquiring, by areceiving terminal of a VR audio, a corresponding first equipmentrotation angle, transmitting the acquired first equipment rotation angleto a transmitting terminal of the VR audio, and/or predicting a secondequipment rotation angle according to the corresponding first equipmentrotation angle and current network delay information, and transmittingthe second equipment rotation angle to the transmitting terminal of theVR audio.

In accordance with another aspect of the disclosure, a receivingterminal equipment for a VR audio is provided. The received terminalequipment includes an acquisition device configured to acquire acorresponding first equipment rotation angle, and at least one processorconfigured to transmit the acquired first equipment rotation angle to atransmitting terminal of a VR audio, and/or predict a second equipmentrotation angle according to the first equipment rotation angle andcurrent network delay information and transmit the second equipmentrotation angle to the transmitting terminal of the VR audio.

In accordance with another aspect of the disclosure, a method forprocessing a VR audio is provided. The method includes acquiring, by atransmitting terminal of a VR audio, an order of a mixed orderambisonics (MOA) signal determined according to related information ofthe VR audio, the related information comprising at least one ofcontent-related information of the VR audio, playback-relatedinformation of the VR audio, and transmission-related information of theVR audio, and extracting, by the transmitting terminal of the VR audio,an MOA signal from an ambisonics signal according to the order of theMOA signal.

In accordance with another aspect of the disclosure, a transmittingterminal equipment for a VR audio is provided. The transmitting terminalequipment includes an acquisition device configured to acquire an orderof a mixed-order ambisonics (MOA) signal determined according to relatedinformation of the VR audio, the related information comprising at leastone of the following, content-related information of the VR audio,playback-related information of the VR audio, and transmission-relatedinformation of the VR audio, and an extraction device configured toextract an MOA signal from an ambisonics signal according to the orderof the MOA signal.

In accordance with another aspect of the disclosure, a method forprocessing a VR audio is provided. The method includes acquiring, by areceiving terminal of a VR audio, related information of the VR audio,the related information comprising at least one of content-relatedinformation of the VR audio, playback-related information of the VRaudio, and transmission-related information of the VR audio, and by thereceiving terminal of the VR audio, transmitting the acquired relatedinformation of the VR audio to a transmitting terminal of the VR audio,or determining an order of an MOA signal according to the acquiredrelated information of the VR audio and transmitting the determinedorder of the MOA signal to the transmitting terminal of the VR audio.

Another aspect of the disclosure is to provide another receivingterminal equipment for a VR audio, comprising an acquisition deviceconfigured to acquire related information of a VR audio, the relatedinformation comprising at least one of content-related information ofthe VR audio, playback-related information of the VR audio, andtransmission-related information of the VR audio, and at least oneprocessor configured to transmit the acquired related information of theVR audio to a transmitting terminal of the VR audio, or determine anorder of an MOA signal according to the acquired related information ofthe VR audio and transmit the determined order of the MOA signal to thetransmitting terminal of the VR audio.

Compared with the prior art, in the method for processing a VR audio andthe corresponding equipment provided by the disclosure, an ambisonicssignal rotation angle can be determined according to a change in theequipment rotation angle corresponding to a receiving terminal of the VRaudio, and an ambisonics is then rotated, so that a terminal equipmentplaying VR audio contents or a terminal equipment playing VR videocontents can still have a very high spatial audio resolution when it isnot placed horizontally.

In addition, in the technical solutions of the disclosure, an order ofan MOA signal determined according to related information of the VRaudio can be acquired, and an MOA signal can be extracted according tothe order of the MOA signal, so that the accuracy of the spatialresolution can be improved and/or the bandwidth occupancy can bereduced.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescriptions taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a schematic diagram of experience of a virtual reality (VR)audio according to the related art;

FIG. 2 is a schematic diagram of the spatial resolution of ambisonicssound fields with different orders according to the related art;

FIG. 3A is a schematic diagram of an ambisonics sound field according tothe related art;

FIG. 3B is a schematic diagram of a mixed-order ambisonics (MOA) soundfield according to the related art;

FIG. 4 is a schematic flowchart of a method for processing a VR audioaccording to an embodiment of the disclosure;

FIG. 5 is a schematic diagram of a gazing plane of a user according toan embodiment of the disclosure;

FIG. 6 is a schematic diagram of a rotation angle of the gazing plane ofthe user according to an embodiment of the disclosure;

FIG. 7 is a schematic flowchart of another method for processing a VRaudio according to an embodiment of the disclosure;

FIG. 8 is a schematic diagram of predicting the rotation angle of thegazing plane of the user according to an embodiment of the disclosure;

FIG. 9 is a schematic diagram of a result of smooth filtering accordingto an embodiment of the disclosure;

FIG. 10 is a schematic flowchart of yet another method for processing aVR audio according to an embodiment of the disclosure;

FIG. 11 is a schematic flowchart of still another method for processinga VR audio according to an embodiment of the disclosure;

FIG. 12 is a schematic flowchart of further another method forprocessing a VR audio according to an embodiment of the disclosure;

FIG. 13 is a schematic flowchart of determining a second equipmentrotation angle according to an embodiment of the disclosure;

FIG. 14 is a schematic flowchart of a preferred method for processing aVR audio according to an embodiment of the disclosure;

FIG. 15 is another schematic flowchart of determining the secondequipment rotation angle according to an embodiment of the disclosure;

FIG. 16 is a schematic flowchart of yet another method for processing aVR audio according to an embodiment of the disclosure;

FIG. 17 is a schematic diagram after rotating according to the gazingplane of the user according to an embodiment of the disclosure;

FIG. 18 is a schematic diagram of the current gazing direction of theuser according to an embodiment of the disclosure;

FIG. 19 is a schematic diagram of the number of virtual loudspeakers inthe horizontal direction according to an embodiment of the disclosure;

FIG. 20 is a schematic diagram of first direction signals and seconddirection signals of 3-order ambisonics according to an embodiment ofthe disclosure;

FIG. 21 is a schematic diagram of the first direction signals and thesecond direction signals according to an embodiment of the disclosure;

FIG. 22 is a schematic diagram of extracting low-order signals accordingto an embodiment of the disclosure;

FIG. 23 is a schematic diagram of combining residual signals and signalsto be transmitted in advance into MOA signals according to an embodimentof the disclosure;

FIG. 24 is a device structure diagram of a transmitting terminalequipment for an VR audio according to an embodiment of the disclosure;

FIG. 25 is a device structure diagram of a receiving terminal equipmentfor an VR audio according to an embodiment of the disclosure;

FIG. 26 is a device structure diagram of another transmitting terminalequipment for an VR audio according to an embodiment of the disclosure;and

FIG. 27 is a device structure diagram of another receiving terminalequipment for a VR audio according to an embodiment of the disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

It should be understood by one person of ordinary skill in the art thatsingular forms “a”, “an”, “the”, and “said” may be intended to includeplural forms as well, unless otherwise stated. It should be furtherunderstood that terms “comprise/comprising” used in this specificationspecify the presence of the stated features, integers, steps,operations, elements and/or components, but not exclusive of thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or combinations thereof. It shouldbe understood that, when a component is referred to as being “connectedto” or “coupled to” another component, it can be directly connected orcoupled to other elements or provided with intervening elementstherebetween. In addition, “connected to” or “coupled to” as used hereincan comprise wireless connection or coupling. As used herein, the term“and/or” comprises all or any of one or more associated listed items orcombinations thereof.

It should be understood by one person of ordinary skill in the art that,unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneperson of ordinary skill in the art to which the disclosure belongs. Itshould be further understood that terms, such as those defined incommonly used dictionaries, should be interpreted as having a meaningthat is consistent with their meanings in the context of the prior artand will not be interpreted in an idealized or overly formal senseunless expressly so defined herein.

It should be understood by one person of ordinary skill in the art thatthe term “terminal” and “terminal equipment” as used herein compassesnot only devices with a wireless signal receiver having no emissioncapability but also devices with receiving and emitting hardware capableof carrying out bidirectional communication over a bidirectionalcommunication link. Such devices can comprise cellular or othercommunication devices with a single-line display or multi-line displayor without a multi-line display; personal communication systems (PCSs)with combined functionalities of speech, data processing, facsimileand/or data communication; personal digital assistants (PDAs), which mayinclude radio frequency (RF) receivers, pagers, internetnetworks/intranet accesses, web browsers, notepads, calendars and/orglobal positioning system (GPS) receivers; and/or laptop of the relatedart and/or palmtop computers or other devices having and/or including aRF receiver. The “terminal” and “terminal equipment” as used herein canbe portable, transportable, mountable in transportations (air, seaand/or land transportations), or suitable and/or configured to runlocally and/or distributed in other places in the earth and/or space forrunning. The “terminal” or “terminal equipment” as used herein may be acommunication terminal, an internet terminal, a music/video playerterminal. For example, it can be a PDA, a mobile Internet device (MID)and/or a mobile phone with a music/video playback function, or can beequipment such as a smart television (TV) and a set-top box.

Embodiment 1

This embodiment of the disclosure provides a method for processing avirtual reality (VR) audio, comprising the following steps, as shown inFIG. 4.

FIG. 4 is a schematic flowchart of a method for processing a VR audioaccording to an embodiment of the disclosure.

Referring to FIG. 4, at operation 401, a transmitting terminal of a VRaudio acquires an ambisonics signal rotation angle, the acquiredambisonics signal rotation angle being determined according to a firstequipment rotation angle corresponding to a receiving terminal of the VRaudio.

In the Embodiment 1 of the disclosure, the receiving terminal of the VRaudio is a terminal equipment receiving VR audio contents. The receivingterminal of the VR audio can comprise a head mount display (HMD)equipment and/or an earphone equipment capable of playing a stereoaudio.

The receiving terminal of the VR audio can receive only the VR audiocontents, and then render and play the received VR audio contents for auser. Furthermore, in addition to receive the VR audio contents, thereceiving terminal of the VR audio can further receive VR video contentscorresponding to the VR audio contents, and synchronously play the VRaudio contents and the VR video contents for a user.

The transmitting terminal of the VR audio is an equipment transmittingthe VR audio contents. The transmitting terminal of the VR audio can bea server having the VR audio contents stored therein, or can be aterminal equipment which is different from the receiving terminal of theVR audio and has the VR audio contents stored therein.

There can be wired connection or wireless connection between thetransmitting terminal equipment and the receiving terminal equipment.

The wireless connection can be at least one of the following connectionmodes: Bluetooth, ultra-wideband, ZigBee, Wireless Fidelity (WiFi)network, general packet radio service (GPRS) network, 3rd-generationwireless telephone technology (3G) network, long-term evolution (LTE)network, or more.

In the Embodiment 1 of the disclosure, the first equipment rotationangle corresponding to the receiving terminal of the VR audio can be anequipment rotation angle of a terminal equipment playing the VR audiocontents (e.g., a rotation angle of an earphone equipment capable ofplaying a stereo audio), or an equipment rotation angle of a terminalequipment playing the VR video contents corresponding to the VR audiocontents (e.g., an equipment rotation angle of an HMD equipment worn bythe user).

The first equipment rotation angle can be measured by a sensor (e.g., aninertia measurement unit). For example, the rotation angle informationcan be obtained according to a result of measurement of the inertiameasurement unit. The first equipment rotation angle can also beobtained by analyzing and calculating images shot by a camera device ofthe equipment. The first equipment rotation angle can be an absolutevalue, or a variable value with respect to an initial angle after systeminitialization. If a variable value with respect to the initial angle isused as the first equipment rotation angle, the initial angle can be anabsolute value, and this value can be transmitted to the transmittingterminal of the VR audio, so that the transmitting terminal of the VRaudio obtains the first equipment rotation angle according to theinitial angle. Wherein, an angle measured when the equipment (theterminal equipment playing the VR audio contents or the terminalequipment playing the corresponding VR video contents) is placedhorizontally can be used as the initial angle after systeminitialization.

FIG. 5 is a schematic diagram of a gazing plane of a user according toan embodiment of the disclosure.

Referring to FIG. 5, in the Embodiment 1 of the disclosure, the firstequipment rotation angle indicates a rotation angle of a gazing planewhen the user listens to the VR audio contents (at this time, the usercan also synchronously watch the VR video contents corresponding to theVR audio contents). As shown in FIG. 5, when the user stands uprightnormally, a plane determined by a straight line to which the line ofsight the eyes of the user gaze corresponds and a straight line thatpasses the user's eyes can be called a gazing plane, or a plane which isparallel to the aforesaid plane and passes both ears can be called as agazing plane. The specific setting to be used depends upon the practicalsituation. As shown in FIG. 5, when the user turns his/her head, thegazing plane of the user will also change, accordingly.

FIG. 6 is a schematic diagram of a rotation angle of the gazing plane ofthe user according to an embodiment of the disclosure.

Referring to FIG. 6, first equipment rotation angles θ, ω, ware rotationangles of the equipment in the x-axis, y-axis and z-axis, and thex-axis, y-axis and z-axis form a space coordinate system using theuser's head as a center. Directions of the x-axis, y-axis and z-axis ofambisonics signals are the same as the directions of those in thiscoordinate system, as shown in FIG. 6. Wherein, the z-axis refers to avertical direction, and the x-axis and y-axis are located in thehorizontal plane.

In the following description of this embodiment of the disclosure, therotation angle of the gazing plane of the user is consistent with theequipment rotation angle.

At operation 402, the transmitting terminal of the VR audio rotates anambisonics signal according to the acquired ambisonics signal rotationangle.

The transmitting terminal of the VR audio extracts a mixed-orderambisonics (MOA) signal from the rotated ambisonics signal, andtransmits the MOA signal to the receiving terminal of the VR audio. Thereceiving terminal of the VR audio renders and plays the received MOAfor the user by itself or other connected equipments. Wherein, the orderof the MOA signal can be determined by an order determination method inthe prior art. For example, the transmitting terminal of the VR audioextracts an MOA signal according to the preset horizontal order andvertical order (for example, horizontal 3-order and vertical 1-order).

The high spatial resolution direction in the existing MOA technology isa fixed horizontal direction and is unable to change with the action ofthe user's head. When the user raises his/her head or performs otheractions, the terminal equipment playing the VR audio contents or theterminal equipment playing the corresponding VR video contents is notplaced horizontally, and the gazing plane of the user is not thehorizontal plane. Thus, the high-order transmission of contents in thehorizontal direction in accordance with the existing MOA technology willreduce the spatial resolution of the sound.

In the method for processing a VR audio provided in Embodiment 1 of thedisclosure, an ambisonics signal rotation angle can be determinedaccording to a change in the equipment rotation angle corresponding tothe receiving terminal of the VR audio (the change in the equipmentrotation angle indicates a change in the rotation angle of the gazingplane of the user), and the ambisonics signal is then rotated, so that aterminal equipment playing VR audio contents or a terminal equipmentplaying corresponding VR video contents can still have a very highspatial audio resolution when it is not placed horizontally (the gazingplane of the user is not horizontal).

Embodiment 2

FIG. 7 is a schematic flowchart of another method for processing a VRaudio according to an embodiment of the disclosure.

The Embodiment 2 of the disclosure is a possible implementation of theEmbodiment 1 of the disclosure. Based on the Embodiment 1 of thedisclosure, the receiving terminal of the VR audio can transmit thefirst equipment rotation angle to the transmitting terminal of the VRaudio, and the transmitting terminal determines an ambisonics signalrotation angle according to the first equipment rotation angle. As shownin FIG. 7, this method comprises the following steps.

At operation 701, the receiving terminal of the VR audio acquires acorresponding first equipment rotation angle.

At operation 702, the receiving terminal of the VR audio transmits thefirst equipment rotation angle to the transmitting terminal of the VRaudio.

The receiving terminal of the VR audio can transmit the original data ofthe first equipment rotation angle to the transmitting terminal of theVR audio. The original data can be rotation angles θ, φ, ω of theequipment in the x-axis, y-axis and z-axis. As required, the receivingterminal of the VR audio can also transmit an angular speed, an angularacceleration or other information to the transmitting terminal of the VRaudio. The angular speed, the angular acceleration or other informationcan be obtained by estimation, or can be obtained by measurement ofequipments.

After the transmitting terminal of the VR audio receives the firstequipment rotation angle information, and if packet loss occurs, theinfluence from the packet loss can be reduced by error concealment. Theerror concealment means that, if the equipment rotation angle data isnot received at the current moment due to the network packet loss orerror, the data at the current moment is estimated by using the datareceived at a historical moment. For example, a value at a previousmoment is used as the value at the current moment, or the lost equipmentrotation angle data is predicted by a prediction algorithm. The errorconcealment step is an optional step, and can reduce the influence fromthe network packet loss.

At operation 703, the transmitting terminal of the VR audio predicts asecond equipment rotation angle according to the received firstequipment rotation angle and current network delay information.

FIG. 8 is a schematic diagram of predicting the rotation angle of thegazing plane of the user according to an embodiment of the disclosure.

In this embodiment of the disclosure, if there is the network delay, thefirst equipment rotation angle (i.e., the rotation angle of the gazingplane) received by the transmitting terminal of the VR audio isdifferent from the equipment rotation angles θ′, φ′ and ω′ at a futuremoment (after the moment T_(delay)). Therefore, it is required topredict the equipment rotation angle, i.e., the second equipmentrotation angle, after a preset network delay time (after the momentT_(delay)) according to the first equipment rotation angle received atthe current moment and the current network delay information, as shownin FIG. 8.

In this embodiment of the disclosure, the network delay T_(delay) can bea sum of the transmission time required for the receiving terminal ofthe VR audio to transmit the related data to the transmitting terminalof the VR audio and the transmission time required for the transmittingterminal of the VR audio to transmit the related data to the receivingterminal of the VR audio.

Wherein, the transmitting terminal of the VR audio can determine secondequipment rotation angles θ′, φ′ and ω′ after the preset network delaytime in the following ways.

Way 1: The transmitting terminal of the VR audio can predict θ′, φ′ andω′ by a predictor, for example, a linear predictor, a Kalman predictoror a Wiener predictor.

Wherein, the linear prediction is as follows: according to θ₁, θ₂, . . .θ_(p) (i.e., rotation angles of the gazing plane about the x-axis) amongthe first equipment rotation angles received at the past p moments, arotation angle θ′ (i.e., a rotation angle of the gazing plane about thex-axis at the current moment) among the first equipment rotation anglesat the current moment is predicted. The processing method for φ and ωare the same as that for θ. The prediction formula is as follows:

$\theta^{\prime} = {\sum\limits_{k = 1}^{p}\; {\beta_{k}\theta_{k}}}$

where β_(k) is a prediction coefficient, and is calculated fromhistorical data; and, the order p of the predictor can be adjustedaccording to the network delay T_(delay), and the formula thereof is

${p = \frac{T_{delay}}{f_{s}}},$

where f_(s) is a sampling frequency of the first equipment rotationangle.

Way 2: If the transmitting terminal of the VR audio can acquire angularspeeds v_(θ), v_(φ), v_(ω) and angular accelerations e_(θ), e_(φ), e_(ω)of the first equipment rotation angle, the first equipment rotationangle after the delay T_(delay) can be calculated in accordance withθ′=v_(θ)T_(delay)+½e_(θ)T_(delay) ², where v_(θ), v_(φ), v_(ω) representangular speeds of the gazing plane rotating about the x-axis, y-axis andz-axis, respectively, and e_(θ), e_(φ), e_(ω) represent angularaccelerations of the gazing plane rotating about the x-axis, y-axis andz-axis, respectively. The processing method for φ and ω is the same asthat for θ.

In this embodiment of the disclosure, in the operation 703, in thepresence of the network delay, the obtained equipment rotation anglesand ambisonics signal rotation angles can be closer to real ones, sothat the spatial resolution will not be reduced due to the networkdelay.

At operation 704, the transmitting terminal of the VR audio determinesan ambisonics signal rotation angle according to the predicted secondequipment rotation angle.

In this embodiment of the disclosure, a reverse angle corresponding tothe second equipment rotation angle can be determined as an ambisonicssignal rotation angle. For example, if the second equipment rotationangle is rotation angles 9, co, co of the equipment about the x-axis,y-axis and z-axis, the final ambisonics signal rotation angle can be −θ,−φ−ω.

At operation 705, the transmitting terminal of the VR audio rotates anambisonics signal according to the rotation angle of the ambisonicssignal.

In the Embodiment 2 of the disclosure, the transmitting terminal of theVR audio rotates the ambisonics signal according to the obtainedrotation angles of the ambisonics signal about the x-axis, y-axis andz-axis, i.e., the rotation angle of the ambisonics signal, so that avery high spatial resolution is still ensured when the gazing plane isnot horizontal.

For example, if the ambisonics signal is a 1-order ambisonics signal,four record channel signals of which are W, X, Y and Z, respectively,where W is an omni-directional recording channel signal, and X, Y and Zare directed to the x-axis, y-axis and z-axis, respectively. Therotation formula is as follows:

[X′Y′Z′]=[XYZ]J

where:

$J = {{\begin{bmatrix}1 & 0 & 0 \\0 & {\cos \left( {- \theta} \right)} & {- {\sin \left( {- \theta} \right)}} \\0 & {\sin \left( {- \theta} \right)} & {\cos \left( {- \theta} \right)}\end{bmatrix}\begin{bmatrix}{\cos \left( {- \phi} \right)} & 0 & {- {\sin \left( {- \phi} \right)}} \\0 & 1 & 0 \\{\sin \left( {- \phi} \right)} & 0 & {\cos \left( {- \phi} \right)}\end{bmatrix}}{\quad\begin{bmatrix}{\cos \left( {- \omega} \right)} & {- {\sin \left( {- \omega} \right)}} & 0 \\{\sin \left( {- \omega} \right)} & {\cos \left( {- \omega} \right)} & 0 \\0 & 0 & 1\end{bmatrix}}}$

is called a rotation matrix, and X′, Y′ and Z′ are rotated X, Y and Zchannel signals.

The transmitting terminal of the VR audio extracts an MOA signal fromthe rotated ambisonics signal, and transmits the MOA signal to thereceiving terminal of the VR audio. The receiving terminal of the VRaudio renders and plays the received MOA signal for the user by itselfor other connected equipments. Wherein, the order of the MOA signal canbe determined by an order determination method in the prior art. Forexample, the transmitting terminal of the VR audio extracts an MOAsignal according to the preset horizontal order and vertical order (forexample, horizontal 3-order and vertical 1-order).

Embodiment 3

The Embodiment 3 of the disclosure is another possible implementation ofthe Embodiment 1 of the disclosure. Based on the Embodiment 2, at leastone of operations 702 a (not shown) and 703 a (not shown) may further beincluded. The steps will be specifically described below.

At operation 702 a, before the operation 702, the receiving terminal ofthe VR audio performs smoothing on the acquired first equipment rotationangle.

In the Embodiment 3 of the disclosure, the receiving terminal performssmoothing on the first equipment rotation angle to eliminate tiny jitterof the user, and the smoothing can be realized by smooth filtering.Wherein, the smooth filtering can be realized by a low-pass filter.

FIG. 9 is a schematic diagram of a result of smooth filtering accordingto an embodiment of the disclosure.

In the Embodiment 3 of the disclosure, the smoothed first equipmentrotation angle can be determined in accordance with the formula{circumflex over (θ)}(n)=Σ_(k=0) ^(K−1)a_(k)θ(n−k), where a_(k) is afilter coefficient, for example, K=3, and a₀=a₁=a₂=⅓; θ(n) denotes therotation angle of the equipment about the x-axis at the moment n; and,{circumflex over (θ)}(n) denotes the smoothed first rotation angle. FIG.9 shows a schematic diagram of the result, and the processing way for φand ω can be determined by the above formula.

At this time, at operation 702, the receiving terminal of the VR audiotransmits the smoothed first equipment rotation angle to thetransmitting terminal of the VR audio. In other words, the firstequipment rotation angle received by the transmitting terminal of the VRaudio is the smoothed first equipment rotation angle.

In the Embodiment 3 of the disclosure, since the first equipmentrotation angle is smoothed, the influence from the jitter noise can beeliminated, so that the accuracy of the subsequently determinedambisonics signal rotation angle can be further improved. When the firstequipment rotation angle remains unchanged after the noise is removed,the receiving terminal can omit the step of transmitting the firstequipment rotation angle to the transmitting terminal, so that theamount of computation is reduced.

At operation 703 a, before the operation 703, the transmitting terminalof the VR audio performs smoothing on the received first equipmentrotation angle.

The specific smoothing way is similar to that in the operation 702 a andwill not be repeated here.

At this time, in the operation 703, the transmitting terminal of the VRaudio predicts a second equipment rotation angle according to thesmoothed first equipment rotation angle and the current network delayinformation.

When the first equipment rotation angle remains unchanged after thenoise is removed, the transmitting terminal can omit the step ofpredicting the second equipment rotation angle, so that the amount ofcomputation is reduced.

It can be seen from the Embodiment 3 of the disclosure that thesmoothing step can be performed by the receiving terminal, or performedby the transmitting terminal, or performed by both the transmittingterminal and the receiving terminal.

Embodiment 4

The Embodiment 4 of the disclosure is another possible implementation ofthe Embodiment 1 of the disclosure. Based on the Embodiment 2 orEmbodiment 3, the operation 704 (the transmitting terminal of the VRaudio determines an ambisonics signal rotation angle according to thepredicted second equipment rotation angle) may further compriseoperations 7041 (not shown) and 7042 (not shown). The steps will bespecifically described below.

At operation 7041, the transmitting terminal of the VR audio performssynthesis according to a weight corresponding to the predicted secondequipment rotation angle and a weight corresponding to the firstequipment rotation angle to obtain the synthesized second equipmentrotation angle.

In the Embodiment 4 of the disclosure, the second equipment rotationangle and the first equipment rotation angle each correspond to therespective weight, respectively, and the transmitting terminal of the VRaudio can perform synthesis based on the weights to obtain thesynthesized second equipment rotation angle.

In addition, a weight corresponding to the first equipment rotationangle θ₁ and a weight corresponding to the second equipment rotationangle θ′ can also be adjusted according to a prediction error rate r,where it is assumed that w₁ represents the weight (also called asynthesized weight) corresponding to θ′, w₂ represents a weightcorresponding to θ₁, 0≤w₁≤1 and w₂=1−w₁.

Wherein, the smaller the prediction error rate r is, the larger thesynthesized weight w₁ is. The synthesis is aimed at reducing theinfluence from the prediction error caused by a prediction model notconforming to the practical situation. The calculation formula for theprediction error rate

$r = \frac{\begin{matrix}{{the}\mspace{14mu} {number}\mspace{14mu} {of}\mspace{14mu} {times}\mspace{14mu} {of}\mspace{14mu} {the}\mspace{14mu} {prediction}} \\{{error}\mspace{14mu} {higher}\mspace{14mu} {than}\mspace{14mu} B\mspace{14mu} \% \mspace{14mu} {within}\mspace{14mu} A}\end{matrix}\mspace{14mu}}{\begin{matrix}{{the}\mspace{14mu} {number}\mspace{14mu} {of}\mspace{14mu} {times}\mspace{14mu} {of}\mspace{14mu} {the}\mspace{14mu} {prediction}} \\{{error}\mspace{14mu} {higher}\mspace{14mu} {than}\mspace{14mu} B\mspace{14mu} \% \mspace{14mu} {within}\mspace{14mu} A}\end{matrix}\mspace{14mu}}$

where B is a preset threshold, A denotes the size of a predictionwindow, and both A and B can be fixed values, or can be determinedexperientially, or can be adjusted in real time.

In this embodiment of the disclosure, the synthesis formula for thesecond equipment rotation angle is θ=w₁θ′+w₂θ₁, where θ is thesynthesized second equipment rotation angle.

Wherein, the calculation formula for the synthesized weight is w₁=i/r,where i is a constant coefficient. Similar processing is performed on φand ω.

In this embodiment of the disclosure, in the operation 7041, theinfluence from the prediction error caused by a prediction model notconforming to the practical situation can be reduced.

In the Embodiment 4 of the disclosure, in operation 7041, the firstequipment rotation angle can be the first equipment rotation angle thatis not smoothed, or the first equipment rotation angle smoothed by thereceiving terminal of the VR audio, or the first equipment rotationangle smoothed by the transmitting terminal of the VR audio. This willnot be limited in this embodiment of the disclosure.

At operation 7042, the transmitting terminal of the VR audio determinesan ambisonics signal rotation angle according to the synthesized secondequipment rotation angle.

Embodiment 5

FIG. 10 is a schematic flowchart of yet another method for processing aVR audio according to an embodiment of the disclosure.

The Embodiment 5 of the disclosure is another possible implementation ofthe Embodiment 1 of the disclosure. Based on the Embodiment 1 of thedisclosure, the receiving terminal of the VR audio can predict a secondequipment rotation angle according to the first equipment rotation angleand then transmit the second equipment rotation angle to thetransmitting terminal of the VR audio. As shown in FIG. 10, this methodcomprises the following steps.

At operation 1001, the receiving terminal of the VR audio acquires acorresponding first equipment rotation angle.

At operation 1002, the receiving terminal of the VR audio predicts asecond equipment rotation angle according to the corresponding firstequipment rotation angle and current network delay information.

Wherein, the specific way of predicting a second equipment rotationangle according to the first equipment rotation angle and the currentnetwork delay information by the receiving terminal of the VR audio isconsistent with the way of predicting the second equipment rotationangle in the operation 703, and will not be repeated here.

At operation 1003, the receiving terminal of the VR audio transmits thesecond equipment rotation angle to the transmitting terminal of the VRaudio.

At operation 1004, the transmitting terminal of the VR audio determinesan ambisonics signal rotation angle according to the received secondequipment rotation angle.

For example, if the second equipment rotation angle is rotation anglesθ, φ, ω of the equipment about the x-axis, y-axis and z-axis, the finalambisonics signal rotation angle can be −θ, −φ, −ω.

At operation 1005, the transmitting terminal of the VR audio rotates anambisonics signal according to the ambisonics signal rotation angle.

After the transmitting terminal of the VR audio receives the secondequipment rotation angle, and if packet loss occurs, the influence fromthe packet loss can be reduced by error concealment. The errorconcealment means that, if the second equipment rotation angle data isnot received at the current moment due to the network packet loss orerror, the data at the current moment is estimated by using the datareceived at a historical moment. For example, a value at a previousmoment is used as the value at the current moment, or the lost secondequipment rotation angle data is predicted by a prediction algorithm.The error concealment step is an optional step, and can reduce theinfluence from the network packet loss.

In the Embodiment 5 of the disclosure, the transmitting terminal of theVR audio rotates the ambisonics signal according to the determinedambisonics signal rotation angle, so that a very high spatial resolutionis still ensured when the gazing plane is not horizontal.

The transmitting of the VR audio extracts an MOA signal from the rotatedambisonics signal, and transmits the MOA signal to the receivingterminal of the VR audio. The receiving terminal of the VR audio rendersand plays the received MOA for the user by itself or other connectedequipments. Wherein, the order of the MOA signal can be determined by anorder determination method in the prior art. For example, thetransmitting terminal of the VR audio extracts an MOA signal accordingto the preset horizontal order and vertical order (for example,horizontal 3-order and vertical 1-order).

Embodiment 6

The Embodiment 6 of the disclosure is another possible implementation ofthe Embodiment 1 of the disclosure. Based on the Embodiment 5, anoperation 1002 a (not shown) may further be included before theoperation 1002, and an operation 1003 a (not shown) may further beincluded before the operation 1003. The steps will be specificallydescribed below.

At operation 1002 a, the receiving terminal of the VR audio performssmoothing on the acquired first equipment rotation angle.

In this embodiment of the disclosure, the way of smoothing the acquiredfirst equipment rotation angle by the receiving terminal of the VR audiois the same as the way of smoothing the first equipment rotation anglein the operation 702 a, and will not be repeated here.

Since the first equipment rotation angle is smoothed, the influence fromthe jitter noise can be eliminated, so that the accuracy of thesubsequently determined second equipment rotation angle and ambisonicssignal rotation angle can be further improved. When the first equipmentrotation angle remains unchanged after the noise is removed, thereceiving terminal can omit the step of predicting the second equipmentrotation angle, so that the amount of computation is reduced.

At operation 1003 a, the receiving terminal of the VR audio performssynthesis according to a weight corresponding to the predicted secondequipment rotation angle and a weight corresponding to the firstequipment rotation angle to obtain the synthesized second equipmentrotation angle.

In this embodiment of the disclosure, the way of synthesizing the secondequipment rotation angle by the receiving terminal of the VR audio isthe same as the way of synthesizing the second equipment rotation angleby the transmitting terminal of the VR audio in the operation 7041, andwill not be repeated here.

At this time, in the operation 1003, the receiving terminal of the VRaudio transmits the synthesized second equipment rotation angle to thetransmitting terminal of the VR audio.

In this embodiment of the disclosure, the second equipment rotationangle synthesized by the receiving terminal of the VR audio can reducethe influence from the prediction error caused by a prediction model notconforming to the practical situation.

Embodiment 7

FIG. 11 is a schematic flowchart of still another method for processinga VR audio according to an embodiment of the disclosure.

The Embodiment 7 of the disclosure is another possible implementation ofthe Embodiment 1 of the disclosure. Based on the Embodiment 1, as shownin FIG. 11, this method comprises the following steps.

At operation 1101, the receiving terminal of the VR audio acquires acorresponding first equipment rotation angle.

At operation 1102, the receiving terminal of the VR audio predicts asecond equipment rotation angle according to the corresponding firstequipment rotation angle and current network delay information.

Wherein, the specific way of predicting a second equipment rotationangle according to the first equipment rotation angle and the currentnetwork delay information by the receiving terminal of the VR audio isconsistent with the way of predicting the second equipment rotationangle by transmitting terminal in the operation 703, and will not berepeated here.

At operation 1103, the receiving terminal of the VR audio transmits thefirst equipment rotation angle and the second equipment rotation angleto the transmitting terminal of the VR audio.

At operation 1104, the transmitting terminal of the VR audio predicts asecond equipment rotation angle according to the received firstequipment rotation angle and the current network delay information.

At operation 1105, the transmitting terminal of the VR audio determinesan ambisonics signal rotation angle according to the received secondequipment rotation angle and the second equipment rotation anglepredicted by itself.

At operation 1106, the transmitting terminal of the VR audio rotates anambisonics signal according to the determined ambisonics signal rotationangle.

After the transmitting terminal of the VR audio receives the firstequipment rotation angle and the second equipment rotation angle, and ifpacket loss occurs, the influence from the packet loss can be reduced byerror concealment. The error concealment means that, if the equipmentrotation angle data is not received at the current moment due to thenetwork packet loss or error, the data at the current moment isestimated by using the data received at a historical moment. Forexample, a value at a previous moment is used as the value at thecurrent moment, or the lost equipment rotation angle data is predictedby a prediction algorithm. The error concealment step is an optionalstep, and can reduce the influence from the network packet loss.

In this embodiment of the disclosure, the transmitting terminal of theVR audio rotates the ambisonics signal according to the determinedambisonics signal rotation angle, so that a very high spatial resolutionis still ensured when the gazing plane is not horizontal.

The transmitting terminal of the VR audio extracts an MOA signal fromthe rotated ambisonics signal, and transmits the MOA signal to thereceiving terminal of the VR audio. The receiving terminal of the VRaudio renders and plays the received MOA for the user by itself or otherconnected equipments. Wherein, the order of the MOA signal can bedetermined by an order determination method in the prior art. Forexample, the transmitting terminal of the VR audio extracts an MOAsignal according to the preset horizontal order and vertical order (forexample, horizontal 3-order and vertical 1-order).

In this embodiment of the disclosure, during the execution of theoperation 1105, the transmitting terminal of the VR audio determines theambisonics signal rotation angle according to at least one of thefollowing information:

a transmission situation of the second equipment rotation angle betweenthe transmitting terminal and the receiving terminal;

a transmission situation of the first equipment rotation angle betweenthe transmitting terminal and the receiving terminal;

a network condition between the transmitting terminal and the receivingterminal; and

the processing capacity of the transmitting terminal and/or thereceiving terminal.

In this embodiment of the disclosure, the network condition between thetransmitting terminal of the VR audio and the receiving terminal of theVR audio may be not stable. That is, packet loss may occur when thetransmitting terminal of the VR audio and the receiving terminal of theVR audio perform signal transmission.

In this embodiment of the disclosure, the receiving terminal of the VRaudio transmits a first equipment rotation angle and a second equipmentrotation angle to the transmitting terminal of the VR audio; when thetransmitting terminal of the VR audio has successfully received thefirst equipment rotation angle but failed to receive the secondequipment rotation angle predicted by the receiving terminal, thetransmitting terminal of the VR audio can determine an ambisonics signalrotation angle according to the second equipment rotation anglepredicted by itself; when the transmitting terminal of the VR audio hassuccessfully received the second equipment rotation angle predicted bythe receiving terminal but failed to receive the first equipmentrotation angle, the transmitting terminal of the VR audio determines anambisonics signal rotation angle according to the second equipmentrotation angle predicted by the receiving terminal; and, when thetransmitting terminal of the VR audio has successfully received both thefirst equipment rotation angle and the second equipment rotation anglepredicted by the receiving terminal, the transmitting terminal of the VRaudio determines an ambisonics signal rotation angle according to theprocessing capacity of the transmitting terminal of the VR audio and/orthe receiving terminal of the VR audio.

In this embodiment of the disclosure, if the processing capacity of thetransmitting terminal of the VR audio is higher than that of thereceiving terminal of the VR audio, the transmitting terminal of the VRaudio can determine an ambisonics signal rotation angle according to thesecond equipment rotation angle predicted by itself; or otherwise, thetransmitting terminal of the VR audio determines an ambisonics signalrotation angle according to the second equipment rotation anglepredicted by the receiving terminal.

As can be seen, in this embodiment of the disclosure, the transmittingterminal and receiving terminal of the VR audio predict a secondequipment rotation angle, respectively, and the transmitting terminal ofthe VR audio determines a final ambisonics signal rotation angleaccording to preset decision conditions.

Wherein, the preset decision conditions are as follows: if the result ofprediction of the receiving terminal of the VR audio fails to bereceived by the transmitting terminal of the VR audio due to the networkpacket loss, and the unpredicted rotation angle data (original data)transmitted by the receiving terminal of the VR audio is correctlyreceived by the transmitting terminal of the VR audio, the result ofprediction of the transmitting terminal of the VR audio is used; if theunpredicted rotation angle data (original data) transmitted by thereceiving terminal of the VR audio fails to be received by thetransmitting terminal of the VR audio due to the network packet loss,and the result of prediction of the receiving terminal of the VR audiois correctly received by the transmitting terminal of the VR audio, theresult of prediction of the receiving terminal of the VR audio is used;and, if no packet loss occurs in the network and if the algorithm forthe transmitting terminal of the VR audio is more complicated andstable, the result of prediction of the transmitting terminal of the VRaudio is used.

In this embodiment of the disclosure, an operation 1102 a (not shown)can be further included before the operation 1102.

At operation 1102 a, the receiving terminal of the VR audio performssmoothing on the acquired first equipment rotation angle.

In this embodiment of the disclosure, the way of smoothing the acquiredfirst equipment rotation angle by the receiving terminal of the VR audiois the same as the way of smoothing the first equipment rotation anglein the operation 702 a, and will not be repeated here.

Since the first equipment rotation angle is smoothed, the influence fromthe jitter noise can be eliminated, so that the accuracy of thesubsequently determined second equipment rotation angle and ambisonicssignal rotation angle can be further improved. When the first equipmentrotation angle remains unchanged after the noise is removed, thereceiving terminal can omit the step of predicting the second equipmentrotation angle, so that the amount of computation is reduced.

In this embodiment of the disclosure, an operation 1103 a (not shown)may further be included before the operation 1103.

At operation 1103 a, the receiving terminal of the VR audio performssynthesis according to a weight corresponding to the predicted secondequipment rotation angle and a weight corresponding to the firstequipment rotation angle to obtain the synthesized second equipmentrotation angle.

In this embodiment of the disclosure, the way of synthesizing the secondequipment rotation angle by the receiving terminal of the VR audio isthe same as the way of synthesizing the second equipment rotation angleby the transmitting terminal of the VR audio in the operation 7041, andwill not be repeated here.

At this time, in the operation 1103, the receiving terminal of the VRaudio transmits the synthesized second equipment rotation angle and thefirst equipment rotation angle to the transmitting terminal of the VRaudio.

In this embodiment of the disclosure, the second equipment rotationangle synthesized by the receiving terminal of the VR audio can reducethe influence from the prediction error caused by a prediction model notconforming to the practical situation.

In this embodiment of the disclosure, an operation 1105 a (not shown)may further be included before the operation 1105.

At operation 1105 a, the transmitting terminal of the VR audio performssynthesis according to a weight corresponding to the second equipmentrotation angle predicted by itself and a weight corresponding to thereceived first equipment rotation angle to obtain the synthesized secondequipment rotation angle.

In this embodiment of the disclosure, the way of synthesizing the secondequipment rotation angle by the transmitting terminal of the VR audio isthe same as the way of synthesizing the second equipment rotation angleby the transmitting terminal of the VR audio in the operation 7041, andwill not be repeated here.

At this time, in the operation 1105, the transmitting terminal of the VRaudio determines an ambisonics signal rotation angle according to thereceived second equipment rotation angle and the synthesized secondequipment rotation angle.

Embodiment 8

FIG. 12 is a schematic flowchart of further another method forprocessing a VR audio according to an embodiment of the disclosure.

This embodiment of the disclosure is a preferred embodiment of thedisclosure, as shown in FIG. 12, wherein:

At operation 1201, a receiving terminal of a VR audio acquires a firstequipment rotation angle and then transmits the first equipment rotationangle to a transmitting terminal of the VR audio.

At operation 1202, the transmitting terminal of the VR audio performserror concealment.

At operation 1203, the transmitting terminal of the VR audio determinesa second equipment rotation angle according to the first equipmentrotation angle and current network delay information.

At operation 1204, the transmitting terminal of the VR audio determinesan ambisonics signal rotation angle according to the second equipmentrotation angle.

At operation 1205, the transmitting terminal of the VR audio rotates anambisonics signal according to the ambisonics signal rotation angle.

At operation 1206, the transmitting terminal of the VR audio extracts anMOA signal from the rotated ambisonics signal.

At operation 1207, the transmitting terminal of the VR audio transmitsthe extracted MOA signal to the receiving terminal of the VR audio.

FIG. 13 is a schematic flowchart of determining a second equipmentrotation angle according to an embodiment of the disclosure.

Wherein, as shown in FIG. 13, the specific process of the operation 1203comprises the following steps.

At operation 1301, the transmitting terminal of the VR audio performssmoothing on the first equipment rotation angle to obtain the smoothedfirst equipment rotation angle.

At operation 1302, the transmitting terminal of the VR audio adjustsparameters of a predictor according to the network delay information.

At operation 1303, the transmitting terminal of the VR audio predicts asecond equipment rotation angle according to the smoothed firstequipment rotation angle and by using the predictor with the adjustedparameters.

At operation 1304, the transmitting terminal of the VR audio synthesizesthe smoothed first equipment rotation angle and the predicted secondequipment rotation angle according to a prediction error rate, to obtainthe synthesized second equipment rotation angle.

FIG. 14 is a schematic flowchart of a preferred method for processing aVR audio according to an embodiment of the disclosure.

Another preferred embodiment of disclosure is provided, as shown in FIG.14, wherein:

At operation 1401, the receiving terminal of the VR audio acquires acorresponding first equipment rotation angle.

At operation 1402, the receiving terminal of the VR audio determines asecond equipment rotation angle according to the first equipmentrotation angle and current network delay information.

At operation 1403, the receiving terminal of the VR audio transmits thedetermined second equipment rotation angle to the transmitting terminalof the VR audio.

At operation 1404, the transmitting terminal of the VR audio performserror concealment.

At operation 1405, the transmitting terminal of the VR audio determinesan ambisonics signal rotation angle according to the second equipmentrotation angle.

At operation 1406, the transmitting terminal of the VR audio rotates anambisonics signal according to the ambisonics signal rotation angle.

At operation 1407, the transmitting terminal of the VR audio extracts anMOA signal from the rotated ambisonics signal.

At operation 1408, the transmitting terminal of the VR audio transmitsthe extracted MOA signal to the receiving terminal of the VR audio.

FIG. 15 is another schematic flowchart of determining the secondequipment rotation angle according to an embodiment of the disclosure.

Wherein, as shown in FIG. 15, the specific process of the operation 1402comprises the following steps.

At operation 1501, the receiving terminal of the VR audio performssmoothing on the first equipment rotation angle to obtain the smoothedfirst equipment rotation angle.

At operation 1502, the receiving terminal of the VR audio adjustsparameters of a predictor according to the network delay information.

At operation 1503, the receiving terminal of the VR audio predicts asecond equipment rotation angle according to the smoothed firstequipment rotation angle and by using the predictor with the adjustedparameters.

At operation 1504, the receiving terminal of the VR audio synthesizesthe smoothed first equipment rotation angle and the predicted secondequipment rotation angle according to a prediction error rate, to obtainthe synthesized second equipment rotation angle.

Embodiment 9

FIG. 16 is a schematic flowchart of yet another method for processing aVR audio according to an embodiment of the disclosure.

Another possible implementation of the various embodiments of thedisclosure comprises the following steps, as shown in FIG. 16.

At operation 1601, a transmitting terminal of a VR audio acquires anorder of an MOA signal determined according to related information ofthe VR audio.

Wherein, the related information of the VR audio comprises at least oneof the following: content-related information of the VR audio,playback-related information of the VR audio, and transmission-relatedinformation of the VR audio.

Wherein, the content-related information of the VR audio comprises atleast one of: content correlation information of the VR audio, soundsource direction information of VR audio contents and VR content typeinformation; the playback-related information of the VR audio comprisesplayback environment noise information, and information about the numberof virtual loudspeakers of the receiving terminal of the VR audio; andthe transmission-related information of the VR audio comprises at leastone of transmission network bandwidth information and transmissionnetwork delay information.

At operation 1602, the transmitting terminal of the VR audio extracts anMOA signal from an ambisonics signal according to the acquired order ofthe MOA signal.

In the Embodiment 9 of the disclosure, the ambisonics signal can be arotated or non-rotated ambisonics signal, and the transmitting terminalof the VR audio can extract an MOA signal from the non-rotatedambisonics signal directly according to the order of the MOA signal uponacquiring the order of the MOA signal.

In this embodiment of the disclosure, the Embodiment 9 can be performednot based on any one of the Embodiments 1 to 8. In other words, theEmbodiment 9 can be performed separately. This will not be limited here.

Although the bandwidth occupancy in the existing MOA technology issomewhat reduced in comparison with the ambisonics technology, inpractical applications, the bandwidth of the MOA technology is still toohigh and the spatial resolution still needs to be improved.

The Embodiment 9 of the disclosure provides a method for processing a VRaudio. Compared with the prior art, in this embodiment of thedisclosure, the receiving terminal of the VR audio acquires an order ofan MOA signal determined according to related information of the VRaudio and then extracts an MOA signal according to the order of the MOAsignal, so that the accuracy of the spatial resolution can be improvedand/or the bandwidth occupancy can be reduced.

Embodiment 10

As another possible implementation of the various embodiments of thedisclosure, based on the Embodiment 9, the operation 1602 of acquiring,by the transmitting terminal of the VR audio, an order of an MOA signaldetermined according to related information of the VR audio may compriseoperations 16021 to 16022 (not shown).

At operation 16021, the transmitting terminal of the VR audio acquiresrelated information of the VR audio.

Operations 16021 a to 16021 b (not shown) may further be included beforethe operation 16022.

At operation 16021 a, the receiving terminal of the VR audio acquiresrelated information of the VR audio.

The related information in the operation 16021 a comprises at least oneof the following: content-related information of the VR audio,playback-related information of the VR audio, and transmission-relatedinformation of the VR audio.

At operation 1602 b, the receiving terminal of the VR audio transmitsthe acquired related information of the VR audio to the transmittingterminal of the VR audio.

At operation 16022, the transmitting terminal of the VR audio determinesan order of an MOA signal according to the acquired related information.

Wherein, the operation 16022 comprises at least one of operations 16022b 1, 16022 b 2 and 16022 b 3, wherein:

At operation 16022 b 1, the transmitting terminal of the VR audiodetermines a total order of the MOA signal according to at least one ofthe VR content type information, the transmission network bandwidthinformation, the transmission network delay information, the playbackenvironment noise information and the information about the number ofvirtual loudspeakers of the receiving terminal.

The transmitting terminal of the VR audio can determine, according tothe determined total order of the MOA signal, a corresponding order ofthe current MOA signal in a first direction and/or a corresponding orderof the current MOA signal in a second direction, and then extract theMOA signal according to the order in the first direction and/or theorder in the second direction.

Wherein, the transmitting terminal of the VR audio can determine,according to a preset principle for allocating the order in the firstdirection and the order in the second direction, the order in the firstdirection and the order in the second direction. In addition, thetransmitting terminal of the VR audio can also adopt a way ofdetermining a corresponding order of the current MOA signal in a firstdirection and/or a corresponding order of the current MOA signal in asecond direction as described in the following embodiments, and this waywill not be repeated here.

Wherein, the first direction can be but not limited to thehorizontal/vertical direction, and the second direction can be but notlimited to the vertical/horizontal direction.

In this embodiment of the disclosure, the transmitting terminal of theVR audio can receive current playback environment noise informationtransmitted by the receiving terminal of the VR audio, and thendetermine the total order of the MOA signal according to the currentplayback environment noise information. In this embodiment of thedisclosure, the playback environment noise information can be acquiredby the receiving terminal of the VR audio. For example, the environmentnoise information can be acquired by a microphone on a mobile phone oron an earphone.

In this embodiment of the disclosure, the receiving terminal of the VRaudio samples the current playback environment noise, determines asampled signal corresponding to each sampling point, and determines theenergy of the current playback environment noise according to thesampled signal corresponding to each sampling point; and, thetransmitting terminal of the VR audio determines the total order of theMOA signal according to the energy of the current playback environmentnoise. In this embodiment of the disclosure, the receiving terminal ofthe VR audio can determine the energy of the current playbackenvironment noise according to the formula:

$E = {\sum\limits_{n = 0}^{N - 1}{s^{2}(n)}}$

where N denotes the number of sampling points required for calculatingthe energy and s(n) denotes the noise signal.

In this embodiment of the disclosure, if the receiving terminal of theVR audio transmits the energy of the playback environment noise to thetransmitting terminal of the VR audio, the transmitting terminal of theVR audio decides the total order of the MOA signal according to theenergy of the playback environment noise. Since the user is difficult toclearly distinguish the position of a sound source in a case in whichthe playback environment noise is higher than the threshold C, the totalorder of the MOA signal can be reduced in the case of high environmentnoise, so that the bandwidth occupancy is reduced.

In this embodiment of the disclosure, the transmitting terminal of theVR audio receives information about the number of virtual loudspeakerstransmitted by the receiving terminal of the VR audio, and determinesthe total order of the MOA signal according to the information about thenumber of virtual loudspeakers. If there are few virtual loudspeakers inthe receiving terminal of the VR audio, the total order of the MOAsignal can be properly reduced, so that the bandwidth occupancy isreduced.

In this embodiment of the disclosure, the receiving terminal of the VRaudio can adjust the number of virtual loudspeakers in the receivingterminal according to at least one of the current gazing direction ofthe user, the current battery level of the receiving terminal of the VRaudio and the computation capability of the receiving terminal of the VRaudio.

FIG. 17 is a schematic diagram after rotating according to the gazingplane of the user according to an embodiment of the disclosure.

In this embodiment of the disclosure, FIG. 17 shows the meaning of anintersection of the user's line of sight and a sphere having a radius R.Wherein, the original coordinate system is rotated along the x-axis andy-axis, respectively, the plane determined by the rotated x-axis andy-axis is a plane where the gazing plane of the user is located, and thepoint A in FIG. 17 denotes the interaction of the line of sight and thesphere having a radius R. If it is assumed that the coordinates of thepoint A in the original coordinate system are [{tilde over (x)}, {tildeover (y)}, {tilde over (z)}], after the coordinate axes are rotated, thecoordinates of the point A in the new coordinate system are as follows:

$\left\lbrack {x,y,z} \right\rbrack = {{\left\lbrack {x,y,z} \right\rbrack \begin{bmatrix}1 & 0 & 0 \\0 & {\cos \mspace{14mu} \theta} & {{- \sin}\mspace{14mu} \theta} \\0 & {\sin \mspace{14mu} \theta} & {\cos \mspace{14mu} \theta}\end{bmatrix}}\begin{bmatrix}{\cos \mspace{14mu} \phi} & 0 & {{- \sin}\mspace{14mu} \phi} \\0 & 1 & 0 \\{\sin \mspace{14mu} \phi} & 0 & {\cos \mspace{14mu} \phi}\end{bmatrix}}$

where θ is a counterclockwise rotation angle of the gazing plane aboutthe x-axis, φ is counterclockwise rotation angle of the gazing planeabout the y-axis, and {umlaut over (z)} calculated according to theformula should be 0.

FIG. 18 is a schematic diagram of the current gazing direction of theuser according to an embodiment of the disclosure.

In this embodiment of the disclosure, the receiving terminal collectscurrent rotation angle information θ, φ, ω of the gazing plane of theuser by a sensor, and then calculates [{umlaut over (x)}, ÿ] accordingto the information. Wherein, the receiving terminal can determine thesight direction of the current gazing plane of the user shown in FIG. 18according to the coordinates [{umlaut over (x)}, ÿ], i.e., the currentgazing direction of the user.

In this embodiment of the disclosure, according to the fact that thehuman ears are more sensitive to the sound in the front than the soundon the right and left and the sound in the rear, and more sensitive tothe sound in the horizontal direction than the sound in the verticaldirection, the receiving terminal of the VR audio can reduce the numberof virtual loudspeakers in at least one of the left and rightdirections, the rear direction and the vertical direction, wherein theformula for two-ear rendering is as follows:

$L = {\sum\limits_{l = 1}^{L}{h_{l}*s_{l}}}$

where h_(l) denotes a head related transfer function (HRTF)corresponding to the l^(th) virtual loudspeaker, * denotes theconvolution, and s_(l) denotes the signal from the i^(th) virtualloudspeaker.

FIG. 19 is a schematic diagram of the number of virtual loudspeakers inthe horizontal direction according to an embodiment of the disclosure.

Wherein, it can be seen from the formula that the amount of computationis less if there are fewer virtual loudspeakers. FIG. 19 shows aschematic diagram of the number of virtual loudspeakers in thehorizontal direction.

In this embodiment of the disclosure, the receiving terminal of the VRaudio acquires the current battery level and/or the computationcapability of the receiving terminal equipment of the VR audio, andfurther adjusts the number of virtual loudspeakers according to thecurrent battery level and/or the computation capability. In thisembodiment of the disclosure, if the batter level of the equipment isinsufficient, it is required to further reduce the number of virtualloudspeakers, according to the fact that the human ears are moresensitive to the sound in the front than the sound on the left and rightand the sound in the rear. In this embodiment of the disclosure, if thenumber of virtual loudspeakers exceeds the computation capability of thereceiving terminal equipment of the VR audio, it is required to furtherreduce the number of virtual loudspeakers.

In this embodiment of the disclosure, the MOA technology has a highrendering power consumption in the client. This is because the amount ofcomputation for two-ear rendering is directly proportional to the numberof virtual loudspeakers and many virtual loudspeakers are generallyrequired during rendering. The receiving terminal of the VR audio canreduce the number of virtual loudspeakers in the above way, so that thepower consumption of the receiving terminal of the VR audio isdecreased.

In this embodiment of the disclosure, the receiving terminal of the VRaudio transmits the adjusted number of virtual loudspeakers to thetransmitting terminal of the VR audio, so that the transmitting terminalof the VR audio determines the total order of the MOA signal accordingto the information about the number of virtual loudspeakers.

In this embodiment of the disclosure, the transmitting terminal of theVR audio determines the total order of the MOA signal according to atleast one of the VR content type information, the transmission networkbandwidth information, the transmission network delay information, theplayback environment noise information and the information about thenumber of virtual loudspeakers in the receiving terminal, and accordingto a weight corresponding to the each kind of information.

In this embodiment of the disclosure, the weight corresponding to theeach kind of information can be predetermined according to experience,or can be adjusted. For example, for a VR content type having higherrequirements on fluency, for example, sports competition programcontent, the weight of the network bandwidth may be larger. When thenetwork bandwidth is insufficient, the total order can be decreased toensure the timeliness of content transmission. However, for musicprogram content, the weight of the network bandwidth may be lower, andan order as high as possible can be used to ensure the audio quality.

For the same VR audio content, if it is assumed that the total order ofthe MOA signal is determined by using only the following conditions i,ii and iv, according to the experience or the settings of a contentprovider, the weight of the condition i can be U_(i)=5, the weight ofthe condition ii can be U_(ii)=3, and the weight of the condition iv canbe U_(iv)=1.

If the content is music content, it is assumed that the total order isP_(i)=10 according to the decision of the condition i. If, in this case,the available network bandwidth is low, the total order is P_(ii)=3according to the decision of the condition ii. If, in this case, thereis low noise, the total order is P_(iv)=9 according to the decision ofthe condition iv. Thus, the total order P is

${P = {\frac{{P_{i}U_{i}} + {P_{ii}U_{ii}} + {P_{iv}U_{iv}}}{U_{i} + U_{ii} + U_{iv}} = {\frac{{10 \times 5} + {3 \times 3} + {9 \times 1}}{5 + 3 + 1} = \frac{68}{9}}}},$

and P is rounded to obtain P=8.

If this content is not music content and the result of decision of theconditions ii and iv is the same as the above result, the total order Pis

${P = {\frac{{P_{ii}U_{ii}} + {P_{iv}U_{iv}}}{U_{ii} + U_{iv}} = {\frac{{3 \times 3} + {9 \times 1}}{3 + 1} = \frac{18}{4}}}},$

and P is rounded to obtain P=5.

In this embodiment of the disclosure, the decision includes two parts:total order decision and decision of an order in a first directionand/or decision of an order in a second direction, which correspond todifferent input conditions. Wherein:

The input conditions corresponding to the total order decision are asfollows:

i. whether the content is audio content of a set type, where the settype can be music or more; the transmitting terminal of the VR audiodecides whether it is a music signal according to a content label, andthe total order is increased as high as possible to improve the spatialresolution if the content is music content;

ii. transmission network bandwidth: if the network bandwidth becomesnarrower, the total order is decreased; or otherwise, the total order isincreased;

iii. transmission network delay: if the network delay is very large, thetotal order can be increased;

in this embodiment of the disclosure, since the direction predictionerror will become larger when the network delay is very large, the ordercan be increased to compensate for the influence from the predictionerror; and

iv. playback environment noise: when the playback environment noise ishigher (or higher than a preset threshold), the total order can bedecreased.

At operation 16022 b 2, the transmitting terminal of the VR audiodetermines, according to at least one of the content correlationinformation of the VR audio and the sound source direction informationof VR audio contents, an order of the MOA signal in a first directionand/or a second direction.

In this embodiment, the transmitting terminal of the VR audio determinescorrelation information of the ambisonics signal in a first directionand/or correlation information of the ambisonics signal in a seconddirection, respectively; and then determines, according to thecorrelation information of the ambisonics signal in the first directionand/or the correlation information of the ambisonics signal in thesecond direction, an order of the MOA signal in the first directionand/or the second direction.

In this embodiment of the disclosure, before the operation 16022 b 2, afinal ambisonics signal rotation direction can be first determined inaccordance with the steps in the Embodiments 1 to 8, the currentambisonics signal is then rotated according to the rotation direction,and correlation information of the rotated ambisonics signal in thefirst direction and/or correlation information of the rotated ambisonicssignal in the second direction is determined, respectively.

FIG. 20 is a schematic diagram of first direction signals and seconddirection signals of 3-order ambisonics according to an embodiment ofthe disclosure.

For example, by taking a 3-order ambisonics signal as example, FIG. 20shows a first signal and a second signal used for determining thecorrelation information in this embodiment of the disclosure.

In this embodiment of the disclosure, the application scenario of is acase in which a sound source in the first direction and a sound sourcein the second direction are very weak in directivity, for example, thebackground music of a TV program.

Wherein, the way of determining the correlation information of theambisonics signal in the first direction and/or the correlationinformation of the ambisonics signal in the second direction will bedescribed below.

In this embodiment of the disclosure, the transmitting terminal of theVR audio determines, according to a sound channel in the firstdirection, correlation information of the ambisonics information in thefirst direction.

Specifically, the correlation information of the ambisonics informationin the first direction and/or the correlation information of theambisonics information in the second direction may be obtained in thefollowing formula A correlation coefficient of X and Y sound channels isas follows:

$\rho_{XY} = \frac{{Cov}\left( {X,Y} \right)}{\sqrt{D(X)}\sqrt{D(Y)}}$

where Cov(X,Y) denotes a covariance, and √{square root over (D(.))}denotes a deviation. The correlation coefficients of the sound channelin the first direction and/or the sound channel in the second directioncan be calculated by pairs. If all correlation coefficients are close to1, it is indicated that the correlation is high; or otherwise, it isindicated that the correlation is low.

In this embodiment of the disclosure, if the correlation of theambisonics signal in the first direction is higher than the correlationof the ambisonics signal in the second direction, the transmittingterminal of the VR audio adjusts the corresponding order of the MOAsignal in the first direction to be higher than the corresponding orderof the MOA signal in the second direction; and, if the correlation ofthe ambisonics signal in the first direction is lower than thecorrelation of the ambisonics signal in the second direction, thetransmitting terminal of the VR audio adjusts the corresponding order ofthe MOA signal in the first direction to be lower than the correspondingorder of the MOA signal in the second direction.

In this embodiment of the disclosure, if the correlation in the firstdirection is high, it is indicated that the signal in the firstdirection has a low directivity, and a lower order can be allocated inthis case; and, if the correlation in the second direction is high, itis indicated that the signal in the second direction has a lowdirectivity, and a lower order can be allocated in this case.

In this embodiment of the disclosure, for a direction with a highcorrelation, a lower order is allocated; while for a direction with alow correlation, a higher order is allocated. In this way, the bandwidthreduction is realized and the spatial resolution is improved.Specifically, the bandwidth occupancy can be reduced while keeping thespatial resolution unchanged; or, the spatial resolution can be improvedwhile keeping the bandwidth unchanged; or, the spatial resolution can beimproved while reducing the bandwidth.

For example, 5-order MOA signals are allocated for the horizontaldirection and 3-order MOA signals are allocated for the verticaldirection (there are total 20 signals). When the horizontal directionhas a low correlation and the vertical direction has a high correlation(it is enough to express the vertical direction by 1-order), 6-ordersignals can be allocated for the horizontal direction and 1-ordersignals can be allocated for the vertical direction (there are total 14signals), so that the spatial resolution can be improved (the resolutionin the horizontal direction is improved and the resolution in thevertical direction remains unchanged, and the overall spatial resolutionis improved) while reducing the bandwidth occupancy. When the horizontaldirection has a high correlation, 4-order signals can be allocated (itis enough to express the horizontal direction by 4-order), and 3-ordersignals are allocated for the vertical direction (there are total 18signals), so that the bandwidth occupancy can be reduced while keepingthe spatial resolution unchanged.

In this embodiment, the transmitting terminal of the VR audio determinessound source energy of the ambisonics signal in a first direction and/orsound source energy of the ambisonics signal in a second direction,respectively; and then determines, according to the sound source energyof the ambisonics signal in the first direction and/or the sound sourceenergy of the ambisonics signal in the second direction, an order of theMOA signal in the first direction and/or the second direction.

In this embodiment of the disclosure, the transmitting terminal of theVR audio acquires a multiple of sound signals in the first direction,and then determines, according to the multiple sound signals in thefirst direction and the corresponding order of the MOA signal in thefirst direction at the current moment, sound source energy of thecurrent MOA signal in the first direction; and/or, the transmittingterminal of the VR audio acquires a multiple of sound signals in thesecond direction, and then determines, according to the multiple ofsound signals in the second direction and the corresponding order of theMOA signal in the second direction at the current moment, sound sourceenergy of the current ambisonics signal in the second direction.

Specifically, if it is assumed that the original ambisonics signal is inK-order, the transmitting terminal of the VR audio determines soundsource energy in the first direction (e.g., the horizontal direction)according to the formula

${E_{H} = \frac{\sum_{k = 1}^{K}{\sum_{n = 0}^{N - 1}\left( {{H_{k}^{\prime 2}(n)} + {H_{k}^{2}(n)}} \right)}}{2K}},$

where H_(k)′ and H_(k) are sound signals in the first direction (e.g.,the horizontal direction).

FIG. 21 is a schematic diagram of the first direction signals and thesecond direction signals according to an embodiment of the disclosure.

Referring to FIG. 21, if the original ambisonics signal is in 3-order,the transmitting terminal of the VR audio determines sound source energyin the horizontal direction according to the formula

$E_{H} = {\frac{\sum_{k = 1}^{3}{\sum_{n = 0}^{N - 1}\left( {{H_{k}^{\prime 2}(n)} + {H_{k}^{2}(n)}} \right)}}{6}.}$

In this embodiment of the disclosure, if it is assumed that the originalambisonics signal is in K-order, the transmitting terminal of the VRaudio determines sound source energy in the second direction (e.g., thevertical direction) according to the formula

${E_{V} = \frac{\sum_{k = 1}^{K}{\sum_{n = 0}^{N - 1}{V_{k}^{2}(n)}}}{K}},$

where V_(k) is a sound signal in the second direction (e.g., thevertical direction).

For example, as shown in FIG. 21, if the original ambisonics signal isin 3-order, the transmitting terminal of the VR audio determines soundsource energy in the vertical direction according to the formula

$E_{V} = {\frac{\sum_{k = 1}^{K}{\sum_{n = 0}^{N - 1}{V_{k}^{2}(n)}}}{K}.}$

In this embodiment of the disclosure, when the sound source energy ofthe ambisonics signal in the first direction is less than the soundsource energy of the ambisonics signal in the second direction, thecorresponding order of the MOA signal in the second direction isincreased, and the corresponding order of the MOA signal in the firstdirection is decreased, so that the spatial resolution can be improvedwithout increasing the bandwidth. In this case, the corresponding orderin the first direction can also be further decreased, so that thespatial resolution is improved under the condition of reducing thebandwidth. Or, the order in the second direction remains unchanged whilethe order in the first direction is decreased, so that the bandwidthoccupancy is reduced under the condition of keeping the spatialresolution unchanged. When the energy of the ambisonics signal in thefirst direction is greater than the energy of the ambisonics signal inthe second direction, the corresponding order of the MOA signal in thesecond direction is decreased, and the corresponding order of the MOAsignal in the first direction is increased, so that the spatialresolution can be improved without increasing the bandwidth.

At operation 16022 b 3, the transmitting terminal of the VR audiodetermines, according to at least one of the VR content typeinformation, the transmission network bandwidth information, thetransmission network delay information, the playback environment noiseinformation and the information about the number of virtual loudspeakersin the receiving terminal, a total order of the MOA signal, anddetermines, according to at least one of the content correlationinformation of the VR audio and the sound source direction informationof VR audio contents, an order of the MOA signal in the first directionand/or the second direction.

The input conditions corresponding to the decision of the order in thefirst direction and/or the second direction are as follows:

v. correlation information of the ambisonics signal in the firstdirection and/or correlation of the ambisonics signal in the seconddirection; and

vi. sound source energy of the ambisonics signal in the first directionand/or sound source energy of the ambisonics signal in the seconddirection.

In this embodiment of the disclosure, in practical applications, thedecision conditions can be freely combined according to actualconditions. For example, when adjusting the total order, the order inthe first direction and the order in the second direction according tothe above conditions, the total order can be adjusted first, and theorder in the first direction and the order in the second direction arethen adjusted. The specific steps are as follows:

a) the total order is decided according to the parameters i, ii, iii andiv, and the result of decision (the total order of the MOA signal) isassumed as z′-order;

b) a ratio of the order in the first direction to the order in thesecond direction is decided according to the parameters v and vi, andthe ratio is assumed as f/g; and

c) the order in the first direction and the order in the seconddirection are calculated according to the formulae

${x^{\prime} = {{\frac{x^{\prime}f}{f + g}\mspace{14mu} {and}\mspace{14mu} y^{\prime}} = \frac{x^{\prime}g}{f + g}}};$

where both x′ and y′ are rounded, to ensure a sum of x′ and y′ to be z′.

Wherein, the ratio of the order in the first direction to the order inthe second direction can also be calculated first, and the total orderis then decided; and then, the order in the first direction and theorder in the second direction are calculated. The specific steps are asfollows:

a) the ratio of the order in the first direction to the order in thesecond direction is decided according to the parameters v and vi, andthe ratio is assumed as f/g;

b) the total order is decided according to parameters i, ii, iii and iv,and the result of decision is assumed as z′-order;

c) the order x′ in the horizontal direction and the order y′ in thevertical direction are calculated according to the following formulas:

$x^{\prime} = \frac{x^{\prime}f}{f + g}$$y^{\prime} = \frac{x^{\prime}g}{f + g}$

d) both x′ and y′ are rounded, to ensure a sum of x′ and y′ to be z′.

In this embodiment of the disclosure, the order is decidedcomprehensively according the above conditions, so that the effect ofreducing the bandwidth occupancy while keeping the spatial resolutionunchanged, or improving the spatial resolution while keeping thebandwidth unchanged, or improving the spatial resolution while reducingthe bandwidth can be realized.

Embodiment 11

As another possible implementation of the various embodiments of thedisclosure, based on the Embodiment 9, the operation 1602 of acquiring,by the transmitting terminal of the VR audio, an order of an MOA signaldetermined according to related information of the VR audio comprisesoperations 16023 to 16024 shown in the Embodiment 10.

At operation 16023, the transmitting terminal of the VR audio receivesan order of the MOA signal determined according to the relatedinformation of the VR audio by the receiving terminal of the VR audio.

In this embodiment of the disclosure, the receiving terminal of the VRaudio can determine a total order of the MOA signal according to theplayback environment noise and/or the information about the number ofvirtual loudspeakers in the receiving terminal of the VR audio, and thespecific way of determining the total order of the MOA signal will notbe repeated here.

At operation 16024, the transmitting terminal of the VR audio determinesa final order of the MOA signal according to the received order of theMOA signal.

Wherein, the operation 16024 comprises operation 16024 b 1 or operation16024 b 2, wherein:

At operation 16024 b 1, the transmitting terminal of the VR audiodetermines the received order of the MOA signal as a final order of theMOA signal.

At operation 16024 b 2, the transmitting terminal of the VR audiodetermines a final order of the MOA signal according to the receivedorder of the MOA signal and the related information of the VR audio.

The transmitting terminal of the VR audio can determine a final order ofthe MOA signal according to at least one other related information,rather than the playback environment noise and the information about thenumber of virtual loudspeakers, in the related information of the VRaudio.

In this embodiment of the disclosure, although the bandwidth occupancyin the MOA technology is somewhat reduced in comparison with theambisonics technology, the bandwidth of the MOA technology is still toohigh in a real-time online browsing scenario. In this embodiment of thedisclosure, the order of the MOA signal can be adjusted according to thecontent-related information, playback-related information andtransmission-related information of the VR audio, so that the bandwidthoccupancy can be reduced and/or the spatial resolution can be improved.

Embodiment 12

As another possible implementation of the various embodiments of thedisclosure, based on the Embodiment 9, operations 1603 to 1604 arefurther included, wherein:

At operation 1603, the transmitting terminal of the VR audio determines,according to the current network state, an order required to betransmitted in advance in the ambisonics signal at a preset moment.

In this embodiment of the disclosure, the transmitting terminal of theVR audio decides, according to the current network state, an order to betransmitted in advance in the ambisonics signal at a future moment d,wherein a higher order to be transmitted in advance is allocated whenthe network state is good; or otherwise, the order to be to transmittedin advance is decreased.

At operation 1604, the transmitting terminal of the VR audio extracts,according to the determined order to be transmitted in advance in theambisonics signal at the preset moment, a signal from the ambisonicssignal at the preset moment in a sequence from a high order to a loworder and according to the determined order required to be transmittedin advance.

In this embodiment of the disclosure, the transmitting terminal of theVR audio extracts, according to the determined order required to betransmitted in advance in the ambisonics signal at the preset moment, alow-order signal from the ambisonics signal at the preset moment, andtransmits the extracted low-order signal to the receiving terminal ofthe VR audio.

Wherein, the low-order refers to the order to be transmitted in advance,and this order is lower than the order of the original ambisonicssignal. For example, if the order of the original ambisonics signal is3-order, 0-order to 2-order belong to a low order. The specificnumerical value M of the low-order is related to the availablebandwidth. If the available bandwidth is higher, the value of M islarger.

FIG. 22 is a schematic diagram of extracting low-order signals accordingto an embodiment of the disclosure.

In this embodiment of the disclosure, the transmitting terminal of theVR audio extracts a low-order signal from the ambisonic signal at themoment d, and transmits the extracted low-order signal to the receivingterminal of the VR audio. Wherein, the method for extracting a low-ordersignal is shown in FIG. 22: extracting, from 3-order ambisonic signals,signals 1, 2, 3, 4, 5, 6, 7, 8 and 9 as low-order (2-order) ambisonicsignals to be transmitted in advance.

At operation 1605, the transmitting terminal of the VR audio transmitsthe extracted signal to the receiving terminal of the VR audio.

At operation 1606, when the preset moment arrives, the transmittingterminal of the VR audio transmits a residual signal rather than theextracted signal in the MOA signal at the preset moment to the receivingterminal of the VR audio.

Wherein, the preset moment is the moment d mentioned above.

In this embodiment of the disclosure, when the preset moment arrives,the transmitting terminal of the VR audio can rotate, according to thefinal ambisonics signal rotation angle, the MOA signal at the presetmoment, then extract a residual signal rather than the extracted signal,and transmit the residual signal and the final ambisonics signalrotation angle to the receiving terminal of the VR audio; or, when thepreset moment arrives, the transmitting terminal of the VR audioextracts a residual signal rather than the extracted signal, andtransmits the residual signal to the receiving terminal of the VR audio.

In this embodiment of the disclosure, when the preset moment arrives,according to at least one of the correlation information of the currentambisonics signal in the first direction and the second direction, thesound source energy of the current ambisonics signal in the firstdirection and the sound source energy of the current ambisonics signalin the second direction, the current environment noise energy, thecontent type information of the current ambisonics signal, the currentnetwork delay information, the current network bandwidth information andthe current environment noise energy information, and the number ofvirtual loudspeakers, the transmitting terminal of the VR audio adjuststhe order of the current MOA signal in the first direction and the orderof the current MOA signal in the second direction. The specific way ofadjusting the order of the MOA signal in the first direction and theorder of the MOA signal in the second direction refers to the variousembodiments described above in detail, and will not be repeated here.

In this embodiment of the disclosure, the transmitting terminal of theVR audio transmits, to the receiving terminal of the VR audio, theresidual signal in the extracted MOA signal, not including the low-ordersignal to be transmitted in advance.

FIG. 23 is a schematic diagram of combining residual signals and signalsto be transmitted in advance into MOA signals according to an embodimentof the disclosure.

For example, if the signal to be transmitted in advance is a 2-ordersignal and the extracted MOA signal is 3-order in the horizontaldirection and 2-order in the vertical direction, the residual signal isshown in FIG. 23.

At operation 1607, when the preset moment arrives, the receivingterminal of the VR audio combines the signal to be transmitted inadvance with the residual signal.

In this embodiment of the disclosure, if the receiving terminal of theVR audio receives the final ambisonics signal rotation angle at thepreset moment in addition to the residual signal transmitted by thetransmitting terminal of the VR audio, the previously received low-orderambisonics signal at the preset moment is rotated according to the finalambisonics signal rotation angle, and the rotated low-order ambisonicssignal at the preset moment is combined with the residual signal; and,if the receiving terminal of the VR audio receives only the residualsignal rather than the signal required to be transmitted in advance inthe MOA signal at the preset moment, the signal to be transmitted inadvance is directly combined with the residual signal.

In this embodiment of the disclosure, when the preset moment arrives,the transmitting terminal of the VR audio rotates the ambisonics signalat the preset moment according to the final ambisonics signal rotationangle, and extracts the MOA signal from the rotated ambisonics signalaccording to the determined order in the first direction and/or theorder in the second direction. In this embodiment of the disclosure, theway of determining the order in the first direction and/or the order inthe second direction is the same as that in the various embodimentsdescribed above, and will not be repeated here.

In this embodiment of the disclosure, since the MOA technology cannotensure that the receiving terminal receives stable audio signals in acase of unstable network, the transmitting terminal of the VR audiotransmits in advance a low-order signal in an MOA signal at a presetmoment to the receiving terminal of the VR audio according to thenetwork state; and when the preset moment arrives, the transmittingterminal of the VR audio transmits a residual signal rather than thetransmitted low-order signal to the receiving terminal of the VR audio.In other words, when the network state is good, a low-order signal isextracted from the ambisonics signal and then transmitted to thereceiving terminal of the VR audio, so that it is ensured that theclient can acquire stable audio signals in the case of unstable network.

FIG. 24 is a device structure diagram of a transmitting terminalequipment for a VR audio according to an embodiment of the disclosure.

An embodiment of the disclosure provides a transmitting terminalequipment for a VR audio. As shown in FIG. 24, the transmitting terminalequipment for a VR audio comprises a first acquisition module 2401(e.g., an acquisition device) and a rotation module 2402 (e.g., arotation device).

The first acquisition module 2401 is configured to acquire an ambisonicssignal rotation angle.

Wherein, the ambisonics signal rotation angle is determined according toa first equipment rotation angle corresponding to a receiving terminalof the VR audio.

The rotation module 2402 is configured to rotate an ambisonics signalaccording to the ambisonics signal rotation angle.

Compared with the prior art, in the transmitting terminal equipment forprocessing a VR audio provided in this embodiment of the disclosure, anambisonics signal rotation angle can be determined according to a changeof the equipment rotation angle corresponding to a receiving terminal ofthe VR audio, and an ambisonics can be then rotated, so that a terminalequipment playing VR audio contents or a terminal equipment playingcorresponding VR video contents can still have a very high spatial audioresolution when it is not placed horizontally.

FIG. 25 is a device structure diagram of a receiving terminal equipmentfor a VR audio according to an embodiment of the disclosure.

An embodiment of the disclosure provides a receiving terminal equipmentfor a VR audio. As shown in FIG. 25, the receiving terminal equipmentfor a VR audio comprises a second acquisition module 2501 and a firstprocessing module 2502 (e.g., at least one processor).

The second acquisition module 2501 is configured to acquire acorresponding first equipment rotation angle.

The first processing module 2502 is configured to transmit the acquiredfirst equipment rotation angle to a transmitting terminal of a VR audio,and/or predict a second equipment rotation angle according to the firstequipment rotation angle and current network delay information andtransmit the second equipment rotation angle to the transmittingterminal of the VR audio.

Compared with the prior art, in the receiving terminal equipment forprocessing a VR audio provided in this embodiment of the disclosure, anambisonics signal rotation angle can be determined according to a changein the equipment rotation angle corresponding to a receiving terminal ofthe VR audio, and an ambisonics can be then rotated, so that a terminalequipment playing VR audio contents or a terminal equipment playingcorresponding VR video contents can still have a very high spatial audioresolution when it is not placed horizontally.

FIG. 26 is a device structure diagram of another transmitting terminalequipment for a VR audio according to an embodiment of the disclosure.

An embodiment of the disclosure provides another transmitting terminalequipment for a VR audio. As shown in FIG. 26, the transmitting terminalequipment for a VR audio comprises a third acquisition module 2601 andan extraction module 2602 (e.g., an extractor).

The third acquisition module 2601 is configured to acquire an order of amixed-order ambisonics (MOA) signal determined according to relatedinformation of the VR audio.

Wherein, the related information comprises at least one of thefollowing: content-related information of the VR audio, playback-relatedinformation of the VR audio, and transmission-related information of theVR audio.

The extraction module 2602 is configured to extract an MOA signal froman ambisonics signal according to the order of the MOA signal.

In the transmitting terminal equipment for processing a VR audioprovided in this embodiment of the disclosure, compared with the priorart, an order of an MOA signal determined according to relatedinformation of the VR audio can be acquired, and an MOA signal is thenextracted according to the order of the MOA signal, so that the accuracyof the spatial resolution can be improved and/or the bandwidth occupancycan be reduced.

FIG. 27 is a device structure diagram of another receiving terminalequipment for a VR audio according to an embodiment of the disclosure.

An embodiment of the disclosure provides another receiving terminalequipment for a VR audio. Referring to FIG. 27, the receiving terminalequipment for a VR audio comprises a fourth acquisition module 2701 anda second processing module 2702 (e.g., at least one processor).

The fourth acquisition module 2701 is configured to acquire relatedinformation of a VR audio.

Wherein, the related information comprises at least one of thefollowing: content-related information of the VR audio, playback-relatedinformation of the VR audio and transmission-related information of theVR audio.

The second processing module 2702 is configured to transmit the acquiredrelated information of the VR audio to a transmitting terminal of the VRaudio, or determine an order of an MOA signal according to the acquiredrelated information of the VR audio and transmit the determined order ofthe MOA signal to the transmitting terminal of the VR audio.

In the receiving terminal equipment for processing a VR audio providedin this embodiment of the disclosure, compared with the prior art, anorder of an MOA signal determined according to related information ofthe VR audio can be acquired, and an MOA signal is then extractedaccording to the order of the MOA signal, so that the accuracy of thespatial resolution can be improved and/or the bandwidth occupancy can bereduced.

The transmitting terminal of the VR audio and the receiving terminal ofthe VR audio provided in the various embodiments of the disclosure areused for implementing the method embodiments described above, and thespecific function implementations refer to the descriptions in themethod embodiments and will not be repeated here. The method forprocessing a VR audio and the corresponding equipment provided in thevarious embodiments of the disclosure can be used to rotate the currentambisonics signal according to the rotation angle of the current gazingplane of the user and then extract an MOA signal from the rotatedambisonics signal according to the determined order of the MOA signal.However, the applications of the method for processing a VR audio andthe corresponding equipment are not limited thereto.

It should be understood by those skilled in the art that the disclosureinvolves devices for carrying out one or more of operations as describedin the disclosure. Those devices can be specially designed andmanufactured as intended, or can comprise well known devices in ageneral-purpose computer. Those devices have computer programs storedtherein, which are selectively activated or reconstructed. Such computerprograms can be stored in device (such as computer) readable media or inany type of media suitable for storing electronic instructions andrespectively coupled to a bus, the computer readable media include butare not limited to any type of disks (including floppy disks, harddisks, optical disks, compact disc red-only memory (CD-ROM) and magnetooptical disks), ROM, random access memory (RAM), erasable programmableROM (EPROM), electrically erasable programmable ROM (EEPROM), flashmemories, magnetic cards or optical line cards. In other words, thereadable media comprise any media storing or transmitting information ina device (for example, computer) readable form.

It should be understood by those skilled in the art that computerprogram instructions can be used to realize each block in structurediagrams and/or block diagrams and/or flowcharts as well as acombination of blocks in the structure diagrams and/or block diagramsand/or flowcharts. It should be understood by those skilled in the artthat these computer program instructions can be provided to generalpurpose computers, special purpose computers or other processors ofprogrammable data processing means to be implemented, so that solutionsdesignated in a block or blocks of the structure diagrams and/or blockdiagrams and/or flow diagrams are executed by computers or otherprocessors of programmable data processing means.

It may be understood by those skilled in the art that the steps,measures and solutions in the operations, methods and flows alreadydiscussed in the disclosure may be alternated, changed, combined ordeleted. Further, other steps, measures and solutions in the operations,methods and flows already discussed in the disclosure can also bealternated, changed, rearranged, decomposed, combined or deleted.Further, the steps, measures and solutions of the prior art in theoperations, methods and operations disclosed in the disclosure can alsobe alternated, changed, rearranged, decomposed, combined or deleted.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details maybe made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A method for processing a virtual reality (VR)audio comprising: determining an ambisonics signal rotation angleaccording to a first equipment rotation angle corresponding to areceiving terminal of the VR audio; acquiring, by a transmittingterminal of a VR audio, the ambisonics signal rotation angle; androtating an ambisonics signal according to the acquired ambisonicssignal rotation angle.
 2. The method for processing a VR audio accordingto claim 1, wherein the acquiring of the ambisonics signal rotationangle comprises at least one of: receiving, by the transmitting terminalof the VR audio, a second equipment rotation angle predicted accordingto a corresponding first equipment rotation angle and current networkdelay information by the receiving terminal of the VR audio, anddetermining an ambisonics signal rotation angle according to thereceived second equipment rotation angle, receiving, by the transmittingterminal of the VR audio, a first equipment rotation angle transmittedby the receiving terminal of the VR audio, predicting a second equipmentrotation angle according to the received first equipment rotation angleand current network delay information, and determining an ambisonicssignal rotation angle according to the predicted second equipmentrotation angle, or receiving, by the transmitting terminal of the VRaudio, a first equipment rotation angle transmitted by the receivingterminal of the VR audio and a second equipment rotation angle predictedaccording to current network delay information, predicting a secondequipment rotation angle according to the received first equipmentrotation angle and the current network delay information, anddetermining an ambisonics signal rotation angle according to thereceived second equipment rotation angle and the second equipmentrotation angle predicted by the transmitting terminal.
 3. The method forprocessing a VR audio according to claim 2, wherein the second equipmentrotation angle received by the transmitting terminal is synthesized bythe receiving terminal according to a weight corresponding to thepredicted second equipment rotation angle and a weight corresponding tothe first equipment rotation angle, respectively, wherein thedetermining of the ambisonics signal rotation angle according to thepredicted second equipment rotation angle comprises: performing, by thetransmitting terminal, synthesis according to the weight correspondingto the predicted second equipment rotation angle and the weightcorresponding to the first equipment rotation angle, and determining anambisonics signal rotation angle according to the synthesized secondequipment rotation angle, and wherein the determining of the ambisonicssignal rotation angle according to the received second equipmentrotation angle and the second equipment rotation angle predicted by thetransmitting terminal comprises: performing, by the transmittingterminal, synthesis according to the weight corresponding to the secondequipment rotation angle predicted by the transmitting terminal and theweight corresponding to the received first equipment rotation angle, anddetermining an ambisonics signal rotation angle according to thereceived second equipment rotation angle and the synthesized secondequipment rotation angle.
 4. The method for processing a VR audioaccording to claim 2, wherein the determining of the ambisonics signalrotation angle according to the received second equipment rotation angleand the second equipment rotation angle predicted by the transmittingterminal comprises: determining, by the transmitting terminal, anambisonics signal rotation angle according to at least one of thefollowing information: a transmission situation of the second equipmentrotation angle between the transmitting terminal and the receivingterminal, a transmission situation of the first equipment rotation anglebetween the transmitting terminal and the receiving terminal, a networkcondition between the transmitting terminal and the receiving terminal,or the processing capacity of the transmitting terminal and/or thereceiving terminal.
 5. The method for processing a VR audio according toclaim 1, further comprising: acquiring, by the transmitting terminal ofthe VR audio, an order of a mixed order ambisonics (MOA) signaldetermined according to related information of the VR audio, the relatedinformation comprising at least one of content-related information ofthe VR audio, playback-related information of the VR audio, ortransmission-related information of the VR audio; and extracting, by thetransmitting terminal of the VR audio, an MOA signal from the rotatedambisonics signal according to the order of the MOA signal.
 6. Themethod for processing a VR audio according to claim 5, wherein thecontent-related information of the VR audio comprises at least one ofcontent correlation information, sound source direction information orVR content type information, wherein the playback-related information ofthe VR audio comprises playback environment noise information, andinformation about the number of virtual loudspeakers in the receivingterminal, and wherein the transmission-related information of the VRaudio comprises at least one of transmission network bandwidthinformation or transmission network delay information.
 7. The method forprocessing a VR audio according to claim 6, wherein the acquiring of theorder of the MOA signal determined according to the related informationof the VR audio comprises: determining, by the transmitting terminal ofthe VR audio, a total order of the MOA signal according to at least oneof the VR content type information, the transmission network bandwidthinformation, the transmission network delay information, the playbackenvironment noise information or the information about the number ofvirtual loudspeakers of the receiving terminal, or determining,according to at least one of the content correlation information of theVR audio and the sound source direction information of VR audiocontents, an order of the MOA signal in a first direction and/or asecond direction.
 8. The method for processing a VR audio according toclaim 5, wherein the acquiring of the order of the MOA signal determinedaccording to related information of the VR audio comprises: receiving,by the transmitting terminal of the VR audio, an order of the MOA signaldetermined according to the related information of the VR audio by thereceiving terminal of the VR audio, and determining a final order of theMOA signal according to the received order of the MOA signal.
 9. Themethod for processing a VR audio according to claim 5, furthercomprising: determining, by the transmitting terminal of the VR audioand according to the current network state, an order required to betransmitted in advance in the ambisonics signal at a preset moment;extracting, by the transmitting terminal of the VR audio and accordingto the determined order required to be transmitted in advance in theambisonics signal at the preset moment, a signal from the ambisonicssignal at the preset moment in a sequence from a low order to a highorder and according to an extracted signal of the determined orderrequired to be transmitted in advance, and transmitting the extractedsignal to the receiving terminal of the VR audio; and when the presetmoment arrives, transmitting a residual signal rather than the extractedsignal in the MOA signal at the preset moment to the receiving terminalof the VR audio.
 10. A transmitting terminal equipment for a virtualreality (VR) audio, comprising: an acquisition device configured toacquire an ambisonics signal rotation angle, the ambisonics signalrotation angle being determined according to a first equipment rotationangle corresponding to a receiving terminal of the VR audio; and arotation device configured to rotate an ambisonics signal according tothe ambisonics signal rotation angle.
 11. A method for processing avirtual reality (VR) audio, characterized in that, comprising the stepsof: acquiring, by a receiving terminal of a VR audio, a correspondingfirst equipment rotation angle; transmitting the acquired firstequipment rotation angle to a transmitting terminal of the VR audio;and/or predicting a second equipment rotation angle according to thecorresponding first equipment rotation angle and current network delayinformation, and transmitting the second equipment rotation angle to thetransmitting terminal of the VR audio.
 12. The method for processing aVR audio according to claim 11, wherein the transmitting of the secondequipment rotation angle to the transmitting terminal of the VR audiocomprises: performing, by the receiving terminal, synthesis according toa weight corresponding to the predicted second equipment rotation angleand a weight corresponding to the first equipment rotation angle, andtransmitting the synthesized second equipment rotation angle to thetransmitting terminal of the VR audio.
 13. The method for processing aVR audio according to claim 11, further comprising: acquiring, by thereceiving terminal of the VR audio, related information of the VR audio,the related information comprising at least one of content-relatedinformation of the VR audio, playback-related information of the VRaudio, or transmission-related information of the VR audio; andtransmitting, by the receiving terminal of the VR audio, the acquiredrelated information of the VR audio to the transmitting terminal of theVR audio, or determining an order of a mixed-order ambisonics (MOA)signal according to the acquired related information of the VR audio andtransmitting the determined order of the MOA signal to the transmittingterminal of the VR audio.
 14. The method for processing a VR audioaccording to claim 13, further comprising: receiving, by the receivingterminal of the VR audio, a signal required to be transmitted in advancein an ambisonics signal at a preset moment transmitted by thetransmitting terminal; and when the preset moment arrives, receiving aresidual signal rather than the signal required to be transmitted inadvance in the MOA signal at the preset moment, and combining the signalrequired to be transmitted in advance with the residual signal.
 15. Themethod for processing a VR audio according to claim 13, furthercomprising: adjusting, according to at least one of the current gazingdirection of a user, the current battery level of the receiving terminalof the VR audio, and the computation capability of the receivingterminal of the VR audio, the number of virtual loudspeakers in thereceiving terminal.
 16. A receiving terminal equipment for a virtualreality (VR) audio, characterized in that, comprising: an acquisitiondevice configured to acquire a corresponding first equipment rotationangle; and at least one processor configured to: transmit the acquiredfirst equipment rotation angle to a transmitting terminal of a VR audio,predict a second equipment rotation angle according to the firstequipment rotation angle and current network delay information, andtransmit the second equipment rotation angle to the transmittingterminal of the VR audio.
 17. A method for processing a virtual reality(VR) audio comprising: acquiring, by a transmitting terminal of a VRaudio, an order of a mixed order ambisonics (MOA) signal determinedaccording to related information of the VR audio, the relatedinformation comprising at least one of content-related information ofthe VR audio, playback-related information of the VR audio, andtransmission-related information of the VR audio; and extracting, by thetransmitting terminal of the VR audio, an MOA signal from an ambisonicssignal according to the order of the MOA signal.
 18. The method forprocessing a VR audio according to claim 17, wherein the content-relatedinformation of the VR audio comprises at least one of contentcorrelation information, sound source direction information or VRcontent type information, wherein the playback-related information ofthe VR audio comprises playback environment noise information, andinformation about the number of virtual loudspeakers in the receivingterminal of the VR audio, and wherein the transmission-relatedinformation of the VR audio comprises at least one of transmissionnetwork bandwidth information or transmission network delay information.19. The method for processing a VR audio according to claim 17, whereinthe acquiring of the order of the MOA signal determined according torelated information of the VR audio comprises: receiving, by thetransmitting terminal of the VR audio, an order of the MOA signaldetermined according to the related information of the VR audio by thereceiving terminal of the VR audio, and determining a final order of theMOA signal according to the received order of the MOA signal.
 20. Themethod for processing a VR audio according to claim 17, furthercomprising: determining, by the transmitting terminal of the VR audioand according to the current network state, an order required to betransmitted in advance in the ambisonics signal at a preset moment;extracting, by the transmitting terminal of the VR audio and accordingto the determined order required to be transmitted in advance in theambisonics signal at the preset moment, a signal from the ambisonicssignal at the preset moment in a sequence from a high order to a loworder and according to the determined order required to be transmittedin advance, and transmitting the extracted signal to the receivingterminal of the VR audio; and when the preset moment arrives,transmitting, by the transmitting terminal of the VR audio, a residualsignal rather than the extracted signal in the MOA signal at the presetmoment to the receiving terminal of the VR audio.